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Make It Count was a large scale, government-funded, project aimed at improving the 
mathematics learning of Indigenous students. NAPLAN Numeracy test results were used as 
one measure of the effect of the program. In this paper we report on the performance on 
these tests of Indigenous students in schools involved in the project. Group data and, where 
available, longitudinal data for individual students are reported. 

After years of debate and considerable publicity, the National Assessment Program - 
Literacy and Numeracy [NAPLAN] was introduced in Australia in 2008. Each year since 
then, NAPLAN tests have been administered nationally to students in Years 3, 5, 7, and 9. 
Formally, participation in the NAPLAN tests is not compulsory. Compliance, however, is 
very high. For example, in 2012 approximately 95% of the Australian Year 3 cohort and 
92% of the Year 9 cohort completed the NAPLAN tests. The numeracy tests contain both 
multiple choice and open-ended items, divided into five different strands: Algebra, 
Function and Pattern; Measurement, Chance and Data; Number; and Space. Data are also 
reported by proficiency band. For each year level, the proportions of students with scores 
in the six proficiency bands deemed appropriate for that year level are provided. For Year 
3, the bands are 1-6; for Year 5, 3-8; for Year 7, 4-9; and for Year 9, bands 5-10. 

Some student background information is also gathered including: student age, gender, 
Indigenous status, language background status (English or non-English), geolocation 
(metropolitan, provincial, remote, and very remote), parental educational background, and 
parental occupation. NAPLAN data are reported overall, separately for these different 
groupings, as well as by state/territory. 

Inspection of the NAPLAN Numeracy data for the years 2008-2012 reveals that, as a 
group, the performance of Indigenous 1 students is consistently below that of non- 
Indigenous students. Addressing this issue, and finding ways to minimize this gap, 
continues to be seen as an urgent educational priority. Footprints in time , which also 
commenced in 2008, is one such initiative: “The main objective of the study is to provide 
high quality quantitative and qualitative data that can be used to provide a better insight 
into how Indigenous children’s early years affect their development” (Footprints in time, 
n.d., p. 87). The focus of this ongoing study is on the first nine or 10 years in the lives of 
Aboriginal and Torres Strait Islander children, and particularly on families living in areas 
of high or extreme isolation. There is no assumption that the findings are representative of 
Australia’s Indigenous population more generally. No serious attempts are therefore made 
to use the data from the study to make comparisons between Indigenous and non- 
Indigenous students. 


1 Indigeneity is determined by self-nomination: “A student is considered to be ‘Indigenous’ if he or she 
identifies as being of Aboriginal and/or Torres Strait Islander origin. The term ‘origin’ is considered to relate 
to people’s Australian Aboriginal or Torres Strait Islander descent and for some, but not all, their cultural 
identity.” (NAPLAN 2012, p. vi) 
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A different longitudinal national project, Make it Count [MiC], is of particular interest 
to those concerned with Indigenous children and the teaching and learning of school-based 
mathematics and numeracy. “Make it Count is an initiative that has developed evidence 
based, responsive mathematics pedagogies to improve the learning outcomes of Aboriginal 
and Torres Strait Islander learners” (AAMT Make It Count, n.d. a). Eight clusters of urban 
and regional schools were involved in the project. Thus findings from the MiC project, too, 
are not necessarily representative of Australia’s wider Indigenous student population, but 
are restricted to those living in metropolitan and provincial areas. 

The Make it Count project 

The nature, scope, and length of the implemented MiC intervention program has varied 
according to the perceived needs of participating schools and their students. In some 
schools all Indigenous students were directly involved in the project; in others this was not 
the case. Evaluation of the practice-driven project was multi-faceted and concerned with 
different features of the program including: student achievement; student experiences in 
mathematics (numeracy); student attitudes, beliefs and self-concept; teacher and school 
change; cultural competency of teachers in schools; and school-community partnerships 
(AAMT Make It Count, n.d. b). In this paper we focus on aspects of the evaluation of 
student achievement. Cautionary remarks made by Yore and Van der Flier-Keller (2011) 
are highly relevant to this endeavour. Referring to their extensive, collaborative project 
aimed at fostering improved performance in mathematics, they concluded that it “made 
significant contributions to high-quality resources and practices, high-quality professional 
learning experiences, and enhanced classroom practices - but it did not provide 
substantive evidence on student performance” (p. 248, emphasis added). 

Setting the context 

Early in the MiC project it was decided that NAPLAN testing results would be used to 
evaluate, or perhaps more accurately monitor, students’ numeracy achievements. 
NAPLAN testing remains contentious. The limitations of the tests are well known and 
were recognised in this study. NAPLAN tests are timed, cover only selected components 
of the mathematics curriculum, and indicate only how well a student performs on the test 
on a given day. Conveniently, however, NAPLAN represents a common test administered 
to all students in Australian schools, and the administration of this uniform measure of 
mathematics assessment is already part of school routines for students in Years 3, 5, 7, and 
9. Reliance on NAPLAN data to monitor progress was therefore considered to be only a 
minimum intrusion into the programs of participating schools. 

As described in the National Reports, NAPLAN tests are equated, enabling “the results 
from NAPLAN tests in different years to be reported on the same achievement scale” 
(NAPLAN, 2012, p. iv). Minor fluctuations in longitudinal test results are expected, and it 
is only when “there is a meaningful change in the results from one year to the next, or 


2 Make it Count was officially launched at the National Indigenous Education Conference in Hobart on 23 
November, 2009 and concluded at the end of 2012. Starting dates of the implementation of the program 
varied by school. The project was funded by the Department of Education, Employment and Workplace 
Relations (DEEWR) and managed by the Australian Association of Mathematics Teachers (AAMT) as part 
of the Closing the Gap - expansion of intensive literacy and numeracy programs initiative. For more details 
of the MiC project see http://makeitcount.aamt.edu.au/ . 
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when there is a consistent trend over several years, that statements about improvement or 
decline in levels of achievement can be made confidently” (NAPLAN, 2012, p. iv). 

Students in the MiC project resided and attended schools in metropolitan and 
regional/provincial areas, but not in remote areas of the country. The data in Table 1, for 
students in Year 3 and for the period 2008-2012, highlight the impact of geolocation on 
mean NAPLAN scores for both Indigenous and non-Indigenous students nationally. There 
are only small variations by year of test administration. The data highlight the large 
performance gap between non-Indigenous and Indigenous students Australia-wide, and 
within each geolocation. The achievement gaps in NAPLAN scores evident for Year 3 are 
replicated at all other year levels, and in each year the tests were administered. 


Table 1 

Year 3 NAPLAN mean scores by Indigeneity and Geolocation 


Geolocation 


2008 

2009 

2010 

2011 

2012 

Australia-wide 

non-Indigenous 

400.5 

397.7 

399.0 

401.7 

399.5 

Indigenous 

327.6 

320.5 

325.3 

334.4 

320.1 

Metro 

non-Indigenous 

404.0 

401.7 

403.0 

406.0 

404.4 

Indigenous 

345.7 

339.7 

343.6 

348.5 

339.4 

Provincial 

non-Indigenous 

392.3 

387.4 

388.4 

390.3 

385.6 

Indigenous 

339.2 

334.3 

336.5 

341.8 

330.7 

Remote 

non-Indigenous 

377.5 

375.3 

380.8 

378.1 

371.9 


Indigenous 

305.7 

287.4 

307 .4 

313.0 

290.8 

Very Remote 

non-Indigenous 

376.1 

371.7 

370.5 

376.0 

367.0 

Indigenous 

265.9 

251.2 

261.4 

286.6 

250.4 

Make it Count 

Indigenous 

340.2 

341.2 

329.9 

332.1 

340.3 


The data in Table 1 indicate that the performance levels on the NAPLAN tests of 
Indigenous students in the MiC schools [referred to as MiC students in the remainder of 
the paper] were generally similar to the mean scores for the national cohorts of Indigenous 
students in Metropolitan or Provincial regions (see shaded sections of Table 1), and were 
higher than the national mean scores of students from remote or very remote areas. A 
comparable pattern of scores was found for MiC students at Years 5, 7, and 9. 

The general pattern is clear. Student achievement levels are directly related to 
geolocation. For each grade level, the further from metropolitan cities schooling takes 
place, the lower is the mean NAPLAN achievement score. Thus, data for Indigenous 
students in metropolitan [Metro] and provincial areas, and not the performance of 
Indigenous students Australia-wide, served as the most appropriate context for the 
performance of MiC students in the evaluation of the MiC program. 

Monitoring progress 

Although evaluation was strongly and consistently emphasized as an important part of 
the MiC project, not all schools submitted the requested information in a usable format. An 
additional complexity in the data gathering arose. MiC schools were involved in the 
program because of their concerns about the mathematics learning of their Indigenous 
students. Yet, over the timespan of the program, as students progressed from one grade 
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level to the next, it was not always clear from the data sent to us which Indigenous students 
were in MiC programs or for how long they were there. Clearly, too, there were changes in 
year level cohorts as individual students moved to, or away from, participating schools. As 
a result, the number of individual Indigenous students for whom we had longitudinal 
NAPLAN data was considerably restricted. It should also be noted that we found that 
particular students were identified as Indigenous in some years and not in others. We are 
not quite sure how this came about, although, as noted in the footnote, Indigeneity is a self- 
nominated category. In what follows, we present two sets of longitudinal data: 

• for Indigenous students in MiC participating schools as a group, and 

• for students in this group for whom we had individual longitudinal data. 

Overall, just over 1450 useable NAPLAN results were submitted. This comprised both 
one-off data from some schools, and repeated information from other schools. The latter 
enabled longitudinal performance patterns to be traced for individual students. 

Longitudinal Group data 

Example 1. NAPLAN mean score differences: For each of the years NAPLAN has been 
administered, we were able to garner NAPLAN data for Indigenous students from the 
schools participating in the MiC project. We also drew data from the National reports of 
NAPLAN performance for Indigenous students across Australia. Space constraints 
preclude displaying the data for all year levels. Presented in Table 2 are Year 3 and Year 9 
mean NAPLAN scores for MiC students from 2008-2012, NAPLAN data for Indigenous 
students nationally and for Metro and Provincial geolocations, and score differences 
between MiC students and the national Indigenous sample overall and for the Metro and 
Provincial sub-groups. 


Table 2 

Year 3 and Year 9 mean NAPLAN scores for MiC Indigenous students, and for Indigenous 
students nationally and by geolocation (Metro/Provincial), and difference scores 




2008 

2009 

2010 

2011 

2012 

Year 3: MiC 

Mean 

340.2 

341.2 

329.9 

332.1 

340.3 


N 

60 

93 

76 

108 

108 

National data: Indigenous students 


327.6 

320.5 

325.3 

334.4 

320.1 

National data: by geolocation 

Metro 

345.7 

339.7 

343.6 

348.5 

339.4 


Provincial 

339.2 

334.3 

336.5 

341.8 

330.7 

Difference scores: 

(MiC mean - National mean) 
Difference scores: 

(MiC mean - Metro mean) 
Difference scores: 

(MiC mean - Provincial mean) 


12.6 

20.7 

4.6 

-2.3 

20.2 

-5.5 

1.5 

-13.7 

-16.4 

0.9 

1.0 

6.9 

-6.6 

-9.7 

9.6 
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2008 

2009 

2010 

2011 

2012 

Year 9: MiC 

Mean 

538.5 

530 

528.7 

528.4 

543.3 


N 

63 

62 

48 

52 

62 

National data: Indigenous students 


515.1 

520.2 

515.2 

515.8 

518.2 

National data: by geolocation 

Metro 

528.0 

531.4 

527.8 

526.8 

528.7 


Provincial 

520.4 

525.5 

520.6 

518.5 

521.5 

Difference scores: 

(MiC mean - National mean) 
Difference scores: 

(MiC mean - Metro mean) 
Difference scores: 

(MiC mean - Provincial mean) 


23.4 

9.8 

13.5 

12.6 

25.1 

10.5 

-1.4 

0.9 

1.6 

14.6 

18.1 

4.5 

8.1 

9.9 

21.8 


Example 2. Gross comparisons between MIC Indigenous students and national 
samples: Since many students attend the same primary school for Years 3 and 5 we could 
expect considerable overlap in the composition of the two groups sitting for the NAPLAN 
Year 3 test in a particular year and the NAPLAN Year 5 test two years later. Given that in 
some states Year 7 is still part of the primary school but in others it is the first year in the 
secondary school, such overlap cannot be assumed as readily for the groups sitting for the 
Year 5 test in a specific year and for the Year 7 test two years later, nor for those sitting for 
a Year 7 test and the Year 9 test two years later. 

A comparison of the performance of Year 3 MiC students in 2008, 2009, and 2010 
with their performance in Year 5 respectively in 2010, 2011, and 2012 is shown in Table 3. 
National results for Indigenous students at the relevant year level and at schools in the 
Metro and Provincial areas are also shown. Of interest are the percentages of students who 
performed ‘below’ or ‘below or at’ the national minimum standards at the pertinent year 
levels since “students who are performing at the National Minimum Standard may also 
require additional assistance to enable them to achieve their potential” (ACARA 2011). 
Band 2 is the national minimum standard at the Year 3 level; at Year 5 it is band 4. 

For each Indigenous cohort, over 10% scored below the national minimum standard; 
for some years, and at some grade levels, the percentage of Indigenous students at MiC 
schools was over 20% (e.g., 2010 Year 5, and 2012 Year 5). For each of the periods 
considered, and for the national Metro and Provincial students as well as the Indigenous 
students at MiC schools, the percentages of students at or below the national minimum 
standards increased from the Year 3 to the Year 5 testing. For example, for Indigenous 
students at MiC schools, the percentage of students at or below the national minimum 
standard was 44.9% in Year 3 in 2010 and rose to 53.9% in Year 5 in 2012. Regrettably, 
these data are difficult to interpret. The different numbers in the groups being compared 
suggest that there was greater fluidity in same-school attendance from Year 3 to Year 5 
than we had anticipated. Nor was it clearly specified which students had, and had not, 
participated in MiC programs. 
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Table 3 

Percentages of students ‘below’ and ‘at or below’ national minimum performance level 
levels for Years 3 and 5: MIC Indigenous students and national samples of Indigenous 
Metro and Provincial students ( exempt students not counted ) 


Group 

N 

% below 
minimum 

% at or below 
minimum 

Group 

N 

% below 
minimum 

% at or below 
minimum 

2008 Year 3 

2010 Year 5 

MiC 

72 

16.7 

40.3 

MiC 

106 

22.6 

60.3 

Metro 


11.7 

36.9 

Metro 


15.9 

42.3 

Prov 


12.5 

40.3 

Prov 


19.4 

49.0 

2009 Year 3 

2011 Year 5 

MiC 

117 

18.8 

41.0 

MiC 

137 

15.3 

45.2 

Metro 


14.5 

40.9 

Metro 


13.7 

42.9 

Prov 


16.8 

43.1 

Prov 


16.3 

47.8 

2010 Year 3 

2012 Year 5 

MiC 

78 

12.8 

44.9 

MiC 

65 

23.1 

53.9 

Metro 


12.5 

37.0 

Metro 


17.3 

44.5 

Prov 


14.5 

42.0 

Prov 


20.5 

49.7 


Longitudinal data for individual students 

Example 3. Individual students’ changes in performance bands: In this section we 
present data for the sample of MiC students for whom we had longitudinal data, that is, 
scores on two successive NAPLAN tests. Shown in Table 4 are results pertaining to Year 
3/Year 5 students in 2008/2010 and Year 3/Year 5 students in 2009/2011 as well as 
corresponding data for students in Year 5/Year 7 and Year 7/Year 9. As noted above, band 
2 is the national minimum standard at the Year 3 level, band 4 at Year 5, band 5 at Year 7, 
and band 6 at Year 9. 

For both sets of Year 3/Year 5 data shown in Table 4, the NAPLAN achievement band 
for approximately half the students (49% and 54%) increased as, or more than, expected. 
The percentage of students (12% and 15%) whose performance increased by more than 2 
bands (that is, higher than expected) did not differ greatly from the percentage (19% and 
14%) whose performance put them in the same or a lower band (that is, their achievement 
bands were lower than expected two years later). 

For the Year 5/Year 7 data, the NAPLAN achievement bands of more than half the 
students (84% and 60%) increased as, or more than, expected. Similarly, for both sets of 
Year 7/Year 9 data, the NAPLAN achievement bands for approximately half the students 
(46% and 58%) also increased as, or more than, expected. The percentages of students who 
stayed at the same band, that is, did not improve after two more years of mathematics 
learning (Year 5/Year 7: 13% and 36%; Year 7/Year 9: 0% and 8%), and the percentages 
of students who moved to a lower band (that is, went backwards) must also be noted. A 
more nuanced analysis of data than possible within the constraints of this paper is needed 
to determine whether all students, or largely only those whose performance met or 
exceeded expectations, were involved in the MiC program. 
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Table 4 

Summary of NAPLAN proficiency band movements for students for whom we had 
longitudinal data: Year 3/Year 5; Year 5/Year 7; and Year 7/Year 9 


Years 

Total 

N 

Up >2 
bands 

Up 2 bands 
(expected) 

Up 1 
band 

Stayed at 
same band 

Moved to a 
lower band 

Year 3 2008 / 

43 

5 

16 

14 

7 

1 

Year 5 2010 

(12%) 

(37%) 

(32%) 

(17%) 

(2%) 

Year 3 2009 / 

72 

11 

28 

23 

6 

4 

Year 5 201 1 

(15%) 

(39%) 

(32%) 

(8%) 

(6%) 



Up> 1 
band 

Up 1 band 
(expected) 


Stayed at 
same band 

Moved to a 
lower band 

Year 5 2008 / 

38 

24 

8 


5 

1 

Year 7 2010 

(63%) 

(21%) 


(13%) 

(3%) 

Year 5 2009 / 

50 

7 

23 


18 

2 

Year 7 2011 

(14%) 

(46%) 


(36%) 

(4%) 

Year 7 2008 / 

13 

0 

6 


7 

0 

Year 9 2010 

(46%) 


(54%) 

Year 7 2009 / 

12 

1 

6 


4 

1 

Year 9 201 1 

(8%) 

(50%) 


(34%) 

(8%) 


Example 4. Changes in performance in mean NAPLAN scores: In Table 5, changes in 
NAPLAN performance over time are in terms of gains in mean scores for MiC Indigenous 
students (for whom we had longitudinal data), and for Indigenous and non-Indigenous 
students nationally and by geolocation (Indigenous only). 

The shaded entries in Table 5 indicate when the gains in mean NAPLAN scores for 
MiC Indigenous students were higher than those of Indigenous students in the Metro and 
Provincial areas nationally (see Year 3/Year 5 in 2009-2011; Year 5/Year 7 in 2008-2010 
and 2009-2011). The consistently low increases in the MiC students’ mean NAPLAN 
scores between Years 7 and 9 are of concern - but note small sample sizes. 

From Table 5 it can also be seen that increases in mean NAPLAN scores at successive 
testings for Indigenous students nationally differ little from those of non-Indigenous 
students (e.g., Year 3/Year 5 in 2009-2011: All non-Indigenous students, gain=93.6; All 
Indigenous students, gain=100.6. Year 5/Year 7 in 2010-2012: All non-Indigenous 
students, gain=49.2; All Indigenous students, gain=52.5). 

As noted earlier in the paper, and illustrated in Table 1, there is a considerable gap in 
the performance at Year 3 of Indigenous and non-Indigenous students nationally. That 
difference in the performance, it seems, remains as the two groups progress through 
school, that is, Indigenous students never close this gap. In some schools, Indigenous 
students in years K-2 were involved in the MiC program. Reliance on NAPLAN scores did 
not allow the mathematical progress of these students to be monitored. 
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Table 5 

Gain in NAPLAN numeracy scores for MiC Indigenous students for whom longitudinal 
data are available, and relevant groups’ national mean score gains 


Period 

MIC 

Mean 

N gain 

Mean gain: 
All Indig 

National data 

Mean gain: Mean gain: 

Indig (Metro) Indig (Prov) 

Mean gain: 

All Non-Indig 

Year 3 to Year 5 cohorts 

2008 - 2010 

36 

66.9 

89.3 

91.1 

88 

92.1 

2009-2011 

57 

109.0 

100.6 

98 

95.6 

93.6 

2010-2012 

32 

88.4 

88.7 

91.3 

89.6 

93.6 

Year 5 to Year 7 cohorts 

2008 - 2010 

28 

81.4 

69.5 

69.0 

67.3 

71.9 

2009-2011 

46 

72.8 

54.3 

52.7 

52.6 

58.2 

2010-2012 

29 

41.0 

52.5 

48.3 

49 

49.2 

Year 7 to Year 9 cohorts 

2008 - 2010 

13 

19.6 

39.0 

34.3 

35.6 

39.9 

2009-2011 

12 

26.3 

41.4 

35.7 

39.8 

39.7 

2010-2012 

25 

14.8 

40.7 

34.8 

36.1 

36.1 


Final words 

Using NAPLAN numeracy scores alone to monitor the performance of MiC students 
yielded mixed results. The improvement in 2012 in the performance of MiC students, 
relative to national data, the substantial percentage of MiC students showing at least 
expected growth in terms of NAPLAN bands, and the modest instances of MiC students 
achieving higher than national gain scores over two-year cycles are heartening. In contrast, 
the persistent high proportion of MiC students at or below the national minimum standard 
and failing to show expected growth in mean NAPLAN scores is disappointing. Further 
analyses, drawing on qualitative data and carefully tracking the involvement in MiC 
programs of students in the early years of schooling and beyond, are clearly warranted. 
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